Time-of-flight (ToF) cameras can sense distance or range to an object by emitting a pulse of modulated light signal and then measuring the time differential in the returning wave front.
A ToF camera can comprise a light source such as a bank of light emitting diodes (LED) whereby a continuously modulated harmonic light signal can be emitted. The distance or range of an object from the light source can be determined by measuring the shift in phase (or difference in time) between the emitted and the reflected photons of light. The reflected photons can be sensed by the camera by the means of charge coupled device or the like.
ToF camera can be combined with one or more 2D cameras to produce a fused image in which a colour image from the 2D camera is combined with the distance or 3D data obtained from the ToF camera.
Typically the ToF camera operates at a lower resolution than that of the 2D camera to which it is combined. Consequently, the fusion of distance 3D data with the colour image requires that data from each of the two cameras are to be aligned. This invariably involves the resampling and projection of the distance three dimensional (3D) data on to a higher resolution grid in order to facilitate fusion with the corresponding colour two dimensional (2D) image.
In particular the repositioning of distance 3D data in order to enable fusion with the colour 2D image can typically take the form of projecting the distance 3D data into a real world coordinate space and then further projecting the data onto a higher resolution grid by resampling.
However, as a consequence of the above projecting and resampling processes any measurement noise in the distance 3D data can significantly deteriorate the quality of the eventual fused image.
fusion of a colour image with the 3D distance data may be particularly sensitive to measurement noise in the distance 3D data resulting in significant deterioration in the quality of the eventual 3D image.
Furthermore, the deterioration in the fused image may be particularly exacerbated (profound) when using low power ToF camera sensing devices, as these devices typically produce distance measurement data which inherently comprises more noise than distance measurement data from a ToF sensor operating in so called normal mode of operation.